The present invention relates to an apparatus and a method for image processing that are suitable for use in converting an interlaced signal including a converted signal converted into an interlaced signal by 3-2 pulldown, 2-2 pulldown or the like and resulting from an editing process into a progressive signal.
Standard video signals such as an NTSC signal, a high-definition television signal and the like are interlaced signals. FIGS. 9A, 9B, and 9C are diagrams showing scanning line structures, FIG. 9A representing an interlaced signal, FIG. 9B representing a progressive signal, and FIG. 9C representing a signal obtained by converting an interlaced signal into a progressive signal by scanning line interpolation. Incidentally, the symbol of a circle (∘) in FIG. 9 represents a scanning line, and the symbol of a cross (x) in FIG. 9 represents an interpolated scanning line.
In FIG. 9, a vertical direction V is the vertical direction of a screen, and a horizontal direction t is a time direction. As shown in FIG. 9A, one frame of an interlaced signal is formed by two fields shifted from each other in the time direction and the vertical direction. On the other hand, there is no shift in the scanning line structure of a progressive signal, as shown in FIG. 9B. In the case of the interlaced signal, an interlace disturbance such as line flicker or the like occurs when a high-frequency component in the vertical direction of an image is increased. On the other hand, the progressive signal is free from the interlace disturbance.
As shown in FIG. 9C, there is a processing method for eliminating the interlace disturbance by interpolating a scanning line in a part discretely reduced by the interlace using neighboring scanning lines, and thereby converting the interlaced signal into a progressive signal. Such a processing method is referred to as progressive conversion or double density conversion.
In the case of an interlaced signal originating from normal video, scanning line interpolation for progressive conversion is performed by a motion adaptive type interpolation process. Specifically, as shown in FIG. 10, when an image is still, a new scanning line is formed by performing inter-field interpolation using an average value of signals PA and PB representing pixels in a previous field and a subsequent field as a signal PQ representing a new pixel denoted by a cross (x) symbol. On the other hand, when an image is moving, a new scanning line is formed by performing intra-field interpolation using an average value of signals PC and PD representing vertically adjacent pixels as the signal PQ representing a new pixel denoted by the cross (x) symbol. Thus, when an image is still, excellent quality of a converted image with little aliasing and high resolution can be obtained. However, when an image is moving, the quality of a converted image is degraded with much aliasing and low resolution.
In a case where an input signal to be converted into a progressive signal includes a converted signal originating from film video which signal is converted by 3-2 pulldown, 2-2 pulldown or the like, excellent converted image quality can be obtained even when the image is moving, by employing a method different from the motion adaptive type interpolation process for the part originating from the film video. The 3-2 pulldown is a frame rate conversion as represented in FIG. 11. Specifically, the 3-2 pulldown is used as a method for converting progressive signals A, B, C . . . (hereinafter referred to as “frames A, B, C . . . ” as appropriate) of film video having a rate of 24 frames per second or the like into interlaced signals a, a′, a, b′, b, c′, c, c′ . . . (hereinafter referred to as “fields a, a′, a . . . ” as appropriate) of an NTSC system having a rate of 60 fields per second. Incidentally, presence or absence of “′” in FIG. 11 indicates a difference between an odd field and an even field. On the other hand, the 2-2 pulldown is a frame rate conversion as represented in FIG. 12. Specifically, the 2-2 pulldown is used as a method for converting progressive signals A, B, C . . . of film video having a rate of 30 frames per second, for example, into interlaced signals a, a′, b, b′, c, c′ . . . of the NTSC system having a rate of 60 fields per second.
As shown in FIG. 11 and FIG. 12, the 3-2 pulldown divides an image as an originally identical frame into three or two fields, while the 2-2 pulldown divides an image as an originally identical frame into two fields. Hence, when the 3-2 pattern or the 2-2 pattern of a part converted into an interlaced signal by 3-2 pulldown or 2-2 pulldown, that is, a pulldown sequence is known, the interlaced signal can be converted into a progressive signal by performing field interpolation using an adjacent field generated from a same frame regardless of whether the image is still or moving. The field interpolation, which is different as an interpolation method from the inter-field interpolation represented in FIG. 10 but is similar to the inter-field interpolation, generates a new scanning line by setting a signal PA in a previous field or a signal PB in a subsequent field as a signal PQ representing a new pixel. Thus, after the conversion, excellent image quality can be obtained with little aliasing and high resolution. Above described image processing apparatus and method in related art is disclosed, for example, in Japanese Patent Laid-Open No. 2004-343333.
There are cases where the pulldown sequence is disrupted as a result of an editing process performed on a part originating from film video which part has been thus converted into an interlaced signal by 3-2 pulldown, 2-2 pulldown or the like. For example, when 3-2 pulldown as represented in FIG. 11 is performed, an interlaced signal normally has a sequence such as fields a, a′, a, b′, b, c′, c, c′, d, and d′. As a result of cutting off the fields c and c′ by an editing process as shown in FIG. 13, the interlaced signal can have a sequence such as fields a, a′, a, b′, b, c′, d, and d′.
When the interlaced signal in which the pulldown sequence is thus disrupted by an editing process is converted into a progressive signal, as a result of field interpolation being performed expecting that fields originating from the frame C such as the fields c′, c, and c′ continue, a frame as indicated by hatch lines in FIG. 13 is generated from the field c′ and the field d before and after a boundary between the field c and the field d, as shown in FIG. 13. However, since the field c′ and the field d originate from different frames on a time axis, the frame generated from the field c′ and the field d is a comb-shaped image, that is, a double image, and thus image quality is greatly impaired.
Incidentally, since such disruption of the sequence is caused by an editing process, the boundary between the field c′ and the field d will hereinafter be referred to as an edit point.
In order to solve such a problem, a progressive conversion unit in related art compares the field c′ and the field d to be subjected to field interpolation with each other, and detects that the field next to the field c′ is the field d rather than the expected field c. However, it is difficult to find an edit point when the field c′ and the field d are similar to each other, for example, when only one part of a screen is moving. Consequently, a frame is generated from the field c′ and the field d, and thus image quality is greatly impaired.